from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.058	0.101	0.0	0.017	-1	1	0.663	1.909	0.033	8.935	0.163	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.058	0.101	0.0	0.017	-1	1	0.663	0.026	0.003	665.626	73.104	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.303	0.007	0.0	0.303	-1	1	1.000	1.909	0.033	0.159	0.005	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.303	0.007	0.0	0.303	-1	1	1.000	0.026	0.003	11.825	1.325	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.040	0.097	0.0	0.017	-1	5	0.757	2.811	0.027	6.062	0.069	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.040	0.097	0.0	0.017	-1	5	0.757	0.025	0.003	670.144	81.923	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.305	0.008	0.0	0.305	-1	5	1.000	2.811	0.027	0.109	0.003	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.305	0.008	0.0	0.305	-1	5	1.000	0.025	0.003	12.010	1.500	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.977	0.062	0.0	0.017	1	100	0.882	2.067	0.012	8.213	0.055	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.977	0.062	0.0	0.017	1	100	0.882	0.025	0.006	683.900	157.831	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.009	0.0	0.304	1	100	1.000	2.067	0.012	0.147	0.005	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.009	0.0	0.304	1	100	1.000	0.025	0.006	12.262	2.855	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.451	0.158	0.0	0.017	-1	100	0.882	2.866	0.035	6.089	0.092	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.451	0.158	0.0	0.017	-1	100	0.882	0.026	0.002	668.277	49.521	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.004	0.0	0.308	-1	100	1.000	2.866	0.035	0.107	0.002	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.004	0.0	0.308	-1	100	1.000	0.026	0.002	11.776	0.877	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.827	0.055	0.0	0.017	1	5	0.757	2.049	0.006	8.211	0.035	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.827	0.055	0.0	0.017	1	5	0.757	0.022	0.000	764.016	7.784	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.006	0.0	0.308	1	5	1.000	2.049	0.006	0.150	0.003	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.006	0.0	0.308	1	5	1.000	0.022	0.000	13.999	0.293	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.959	0.024	0.0	0.017	1	1	0.663	1.185	0.004	14.307	0.049	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.959	0.024	0.0	0.017	1	1	0.663	0.022	0.002	780.284	82.739	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.307	0.006	0.0	0.307	1	1	1.000	1.185	0.004	0.259	0.005	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.307	0.006	0.0	0.307	1	1	1.000	0.022	0.002	14.130	1.525	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.711	0.015	0.0	0.004	-1	1	0.896	1.776	0.020	2.090	0.025	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.711	0.015	0.0	0.004	-1	1	0.896	0.006	0.004	617.247	398.997	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.005	0.0	0.242	-1	1	1.000	1.776	0.020	0.136	0.003	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.005	0.0	0.242	-1	1	1.000	0.006	0.004	40.255	26.032	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.660	0.009	0.0	0.004	-1	5	0.922	2.685	0.029	1.363	0.015	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.660	0.009	0.0	0.004	-1	5	0.922	0.009	0.002	391.799	78.498	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.005	0.0	0.242	-1	5	1.000	2.685	0.029	0.090	0.002	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.005	0.0	0.242	-1	5	1.000	0.009	0.002	25.931	5.220	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.723	0.010	0.0	0.004	1	100	0.929	1.959	0.001	1.900	0.005	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.723	0.010	0.0	0.004	1	100	0.929	0.003	0.000	1294.527	129.320	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	1	100	1.000	1.959	0.001	0.122	0.002	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	1	100	1.000	0.003	0.000	83.357	8.445	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.705	0.010	0.0	0.004	-1	100	0.929	2.826	0.064	1.311	0.030	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.705	0.010	0.0	0.004	-1	100	0.929	0.010	0.003	356.861	101.027	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	-1	100	1.000	2.826	0.064	0.085	0.002	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	-1	100	1.000	0.010	0.003	23.147	6.566	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.669	0.014	0.0	0.004	1	5	0.922	1.950	0.003	1.882	0.008	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.669	0.014	0.0	0.004	1	5	0.922	0.003	0.000	1320.398	98.992	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.005	0.0	0.241	1	5	1.000	1.950	0.003	0.123	0.002	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.005	0.0	0.241	1	5	1.000	0.003	0.000	86.657	6.717	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.713	0.018	0.0	0.004	1	1	0.896	1.081	0.003	3.436	0.019	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.713	0.018	0.0	0.004	1	1	0.896	0.002	0.000	2156.451	304.990	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	1	1	1.000	1.081	0.003	0.223	0.004	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	1	1	1.000	0.002	0.000	139.956	19.944	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	120.845	0.000	0.0	0.121	-1	1	0.929	0.469	0.004	257.751	2.157	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	120.845	0.000	0.0	0.121	-1	1	0.929	0.003	0.001	41697.915	7452.127	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.615	0.144	0.0	2.615	-1	1	1.000	0.469	0.004	5.578	0.311	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.615	0.144	0.0	2.615	-1	1	1.000	0.003	0.001	902.359	168.784	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	122.155	0.000	0.0	0.122	-1	5	0.946	0.887	0.004	137.791	0.622	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	122.155	0.000	0.0	0.122	-1	5	0.946	0.003	0.001	37824.971	6659.353	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.581	0.109	0.0	2.581	-1	5	1.000	0.887	0.004	2.911	0.123	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.581	0.109	0.0	2.581	-1	5	1.000	0.003	0.001	799.107	144.645	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	130.098	0.000	0.0	0.130	1	100	0.951	5.374	0.008	24.209	0.037	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	130.098	0.000	0.0	0.130	1	100	0.951	0.003	0.001	38117.155	9039.924	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.608	0.163	0.0	2.608	1	100	1.000	5.374	0.008	0.485	0.030	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.608	0.163	0.0	2.608	1	100	1.000	0.003	0.001	764.018	187.371	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	133.856	0.000	0.0	0.134	-1	100	0.951	2.953	0.030	45.323	0.453	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	133.856	0.000	0.0	0.134	-1	100	0.951	0.006	0.001	21866.706	3569.375	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.574	0.061	0.0	2.574	-1	100	1.000	2.953	0.030	0.871	0.022	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.574	0.061	0.0	2.574	-1	100	1.000	0.006	0.001	420.434	69.353	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	135.558	0.000	0.0	0.136	1	5	0.946	1.620	0.004	83.683	0.231	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	135.558	0.000	0.0	0.136	1	5	0.946	0.002	0.000	89871.803	25730.382	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.602	0.116	0.0	2.602	1	5	1.000	1.620	0.004	1.606	0.072	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.602	0.116	0.0	2.602	1	5	1.000	0.002	0.000	1724.853	499.807	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	140.781	0.000	0.0	0.141	1	1	0.929	0.848	0.003	166.013	0.643	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	140.781	0.000	0.0	0.141	1	1	0.929	0.001	0.000	143673.194	40400.996	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.585	0.059	0.0	2.585	1	1	1.000	0.848	0.003	3.048	0.070	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.585	0.059	0.0	2.585	1	1	1.000	0.001	0.000	2637.847	744.189	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.037	0.000	0.0	0.000	-1	1	0.891	0.022	0.001	1.724	0.092	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.037	0.000	0.0	0.000	-1	1	0.891	0.002	0.000	15.181	2.129	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.022	0.001	0.237	0.015	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.002	0.000	2.084	0.299	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	-1	5	0.911	0.022	0.001	1.718	0.062	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	-1	5	0.911	0.002	0.000	17.280	2.290	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.022	0.001	0.231	0.010	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.002	0.000	2.319	0.314	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.000	0.0	0.000	1	100	0.894	0.032	0.000	1.841	0.020	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.000	0.0	0.000	1	100	0.894	0.001	0.000	98.388	26.149	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.032	0.000	0.160	0.005	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.001	0.000	8.560	2.286	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.000	0.0	0.000	-1	100	0.894	0.034	0.001	1.775	0.064	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.000	0.0	0.000	-1	100	0.894	0.002	0.000	29.593	6.256	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.001	0.0	0.005	-1	100	1.000	0.034	0.001	0.158	0.016	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.001	0.0	0.005	-1	100	1.000	0.002	0.000	2.626	0.609	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	5	0.911	0.020	0.000	1.964	0.027	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	5	0.911	0.001	0.000	64.445	19.168	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.020	0.000	0.264	0.008	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.001	0.000	8.650	2.582	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	1	0.891	0.020	0.004	1.897	0.346	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	1	0.891	0.001	0.000	64.556	17.172	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.020	0.004	0.256	0.047	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.001	0.000	8.711	2.332	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	n_iter	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.464	0.015	300	0.002	0.000	0.824	0.107	0.002	4.317	0.162	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.464	0.015	300	0.002	0.000	0.824	0.016	0.001	28.260	1.265	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	0.386	0.009	300	0.000	0.386	1.000	0.107	0.002	3.587	0.114	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	0.386	0.009	300	0.000	0.386	1.000	0.016	0.001	23.485	0.937	See	See